Papers with generative replay
Generative Replay Inspired by Hippocampal Memory Indexing for Continual Language Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Continual learning (CL) is a fundamental requirement for human-like general intelligence (Parisi et al., 2019). |
| Approach: | They propose to control sample generation using compressed features of previous training samples by using hippocampal memory indexing to enhance the generative replay. |
| Outcome: | The proposed method outperforms current generative replay methods and generates training samples from previous tasks. |
Prompt Augmented Generative Replay via Supervised Contrastive Learning for Lifelong Intent Detection (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to identify all possible user intents at design time are expensive and require storage of past data. |
| Approach: | They propose to continually train an intent detector on new intents while maintaining performance on prior intents. |
| Outcome: | The proposed method outperforms exemplar replay-based approaches on lifelong intent detection tasks and achieves state-of-the-art on four public datasets. |
Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialogue (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing generative replay methods use only a single task-specific token to control their models. |
| Approach: | They propose a method to capture task-specific distributions with a conditional variational autoencoder, conditioned on natural language prompts to guide the pseudo-sample generation. |
| Outcome: | The proposed method outperforms baselines on natural language understanding tasks of advanced task-oriented dialogue (ToD) systems. |